LIBRARY OF CONGRESS 



021 338 617 7 









No. 116 



THE MEASUREMENT OF EDUCA- 
TIONAL PROCESSES AND 
PRODUCTS 



LEONARD P. AYRES, Ph.D. 



RS 
.p. 



Address Delivered Before the 

Harvard Teachers' Association, Cambridge, Mass. 

March 9, 1912 



Division of Education 

Russell Sage Foundation 

400 Metropolitan Tower, New York City 






The Measurement of Educational Processes 
and Products 

Fifteen years ago the school superintendents of America, as- 
sembled in convention in Indianapolis, discussed the problems 
then foremost in educational thought and action. At that meet- 
ing a distinguished educator — the pioneer and pathfinder among 
the scientific students of education in America — brought up for 
discussion the results of his investigations of spelling among the 
children in the school systems of nineteen cities. These results 
showed that, taken all in all, the children who spent forty minutes 
a day for eight years in studying spelling did not spell any better 
than the children in the schools of other cities where they devoted 
only ten minutes per day to the study. 

The presentation of these data threw that assemblage into 
consternation, dismay, and indignant protest. But the resulting 
storm of vigorously voiced opposition was directed not against 
the methods and results of the investigation, but against the 
investigator who had pretended to measure the results of teach- 
ing spelling by testing the ability of the children to spell. 

In terms of scathing denunciation the educators there pres- 
ent and the pedagogical experts, who reported the deliberations 
of the meeting in the educational press, characterized as silly, 
dangerous, and from every viewpoint reprehensible, the attempt 
to test the efficiency of the teacher by finding out what the pupils 
could do. With striking unanimity they voiced the conviction 
that any attempt to evaluate the teaching of spelling in terms of 
the ability of the pupils to spell was essentially impossible and 
based on a profound misconception of the function of education. 

Last week in the city of St. Louis that same association of 
school superintendents, again assembled in convention, devoted 
forty-eight addresses and discussions to tests and measurements 
of educational efficiency. The basal proposition underlying this 
entire mass of discussion was that the effectiveness of the school, 
the methods, and the teachers must be measured in terms of the 
results secured. 

i 



This change represents no passing fad or temporary whim. 
It is permanent, significant, and fundamental. It means that 
a transformation has taken place in what we think as well as in 
what we do in education. It means that education is emerging 
from among the vocations and taking its place among the pro- 
fessions. 

This profound change in our educational practice has not come 
through the slow processes of philosophy, or because we were 
awakened by the stirring words of voice or pen of any educational 
prophet. Few schoolmen can claim great credit for having 
hastened its advent. It was forced upon us, first, by the natural 
results of compulsory education, and still more definitely and 
directly by the exactions of the scientific age in which we live. 

The Scientific Method in Education and Industry 
This new attitude of educators towards education means that 
we have ceased exalting the machinery and have commenced to 
examine the product. We have awakened to a startled realiza- 
tion that in education, as in other forms of organized activity, 
applied science may avail to better even those processes that have 
rested secure in the sanction of generations of acceptance. 

The transformation now taking place in education means that 
it is our privilege to be part of a movement that is working 
changes comparable to those that are now remaking almost 
every form of industrial activity. The trade of bricklaying, 
practiced by millions of intelligent artisans, has remained al- 
most unaltered since the days of primitive man. But scientific 
management steps in and asks, Why lower a hundred pounds of 
human flesh to pick up each two pound brick? Why toss the 
brick four times to find its best face? Why tap it three times to 
get its proper level? Why stand in a position that requires half 
a dozen movements when one will suffice? And science makes 
answer: — Build a platform for the bricks adjusted to the height 
of the work; lay the bricks on the platform with the best face 
out; mix the mortar so that one tap will suffice; and take such a 
position that five movements accomplish the same results that 
formerly required eighteen. The result is that each workman 
lays each hour as many bricks as he formerly laid in three hours. 
The ideals and processes of the application of the scientific 
method to education are in salient respects similar to those that 
are reshaping the processes of industry. In education as in in- 



dustry the scientific idea is at base analytic scrutiny, exact 
measuring, careful recording, and judgment on the basis of 
observed fact. Swiftly, silently, and almost without warning, 
the scientific methods have invaded the educational camp and 
have begun to demolish the hosts of theory, legend, supersti- 
tion, and tradition. 

The time has already passed for us to query whether or not we 
shall endorse and adopt the new scientific criteria of exact meas- 
urement and judgment by results. The new method is upon us, 
and the question at issue is no longer "Shall we adopt it?" but 
rather "How shall we utilize it?" 

Reform in Educational Administration 

Three years ago twenty-nine cities in America had systems of 
individual record cards for keeping the school histories of their 
children. Today 216 cities have adopted a uniform system for 
this purpose. Those cities intend to judge processes by results. 

One year ago the number of city school systems having uniform 
records of accounting whereby the school facts of one locality 
could be compared with those of another was about fifteen. 
Today the number of such cities is 418. Their aim is a mutual 
comparison of results. 

Seven years ago Superintendent Maxwell of New York City 
published data in his annual report showing that 39 per cent of 
the school children of that city were above the normal ages for 
their grades. Judged by the age standards they were educa- 
tional misfits. At that time these data were almost unique and 
attracted widespread attention because of their revolutionary 
character. Today such methods of checking up the results of 
our school work are commonplace, and a few months ago the 
Federal Bureau of Education published similar data showing 
conditions in 318 cities. 

These nation-wide changes are not products of mere chance. 
They have come because the oublic and the educators have begun 
to demand real information about their public schools. Less 
than five years ago it occurred to a few people in America seriously 
to ask the question, "What proportion of the children who enter 
our common schools remain to complete the course?" This was 
a plain business proposition. The question at issue was the 
relation of the finished product to the raw material. The chil- 
dren who enter our public schools in the first grade are the raw 



4 
material; those who complete the course and graduate are the 
finished product. It was an elementary question in business 
administration that these students were asking when they in- 
quired what proportion of the children complete the common 
school course. 

In order to answer this question we must have two figures. 
First, the number of children who graduate. That can easily be 
ascertained in any school system. Second, the number of chil- 
dren who begin school each year. That cannot be obtained so 
easily. Incredible as it may seem, up to five years ago schoolmen 
had never thought it worth while to record that datum. A 
patient search showed that the cities in America recording the 
number of children entering school each year could be counted 
on the thumbs of two hands. Today the number of cities keep- 
ing such records runs into the scores. 

At that time the school superintendents knew little more about 
the matter than that the beginners were numerous; that progress 
was not uncommon ; and that there were some graduates each 
year. Now they know that in the country as a whole not one- 
half of the children who enter the public schools remain to grad- 
uate, and they are busily at work remaking their school systems 
to remedy that condition. 

The startling revelation that our vaunted system of free educa- 
tion was failing to give even complete elementary schooling to a 
majority of the children evoked imperious demands for more real 
facts. Here were statements of educational conditions within 
the comprehension of all and painfully obvious in their signifi- 
cance. They left no room for question as to the necessity for 
checking up results in education. 

The school children are the invested capital of the community. 
What should we say of a bank that kept its accounts in the same 
way that the school has kept account of the invested capital of 
society? What would you say if your banker should confess that 
the only facts revealed by his books were the total number of 
accounts handled during the year and the average monthly 
assets? What would you say if he should confess that he did 
not know and could not find out anything about the number or 
amount of new accounts received, old ones withdrawn, or the 
results of his investments? 

Nor was this situation confined to elementary schools. Con- 
ditions in our higher schools were even more shocking. The 



5 
relation between the pupils who entered and those who finished 
was startlingly small. We can hardly imagine an analogous 
situation in any commercial industry. What, for example, 
should we say of a four-act play in the theatre where a thousand 
people were present at the beginning of the first act, five hundred 
got up and left before the beginning of the second, two hundred 
and fifty of these refused to sit through the third, and only one 
hundred and twenty-five remained to see the final descent of the 
curtain? And yet these figures express conditions in many of 
our larger cities with respect to the falling out of pupils in the 
four years of our high school courses. 

Authority versus Evidence 

The new method which judges processes in terms of results 
has been by no means confined to the development of record forms 
and the perfecting of new devices in the statistics of school ad- 
ministration. 

About three years ago a graduate student in one of the uni- 
versities of Massachusetts tried to investigate the old problem 
"What is the best age to send a child to school?" In his search 
for information he put the question to the head of every college 
department of pedagogy in this country. He received definite 
and positive replies from almost all to the effect that the best 
entering age is a comparatively late one. He then followed his 
first inquiry by a second in which each pedagogical expert was 
asked on what he based his assertion. In every case save one 
the answer was that the writer was positive of the correctness of 
his views, but had no evidence with which to substantiate them. 
The exception was a man who said that he knew because his 
own son had entered school late and had made good progress. 

This happened only three years ago, and the answers were 
speculative and indefinite because quantitative evidence bearing 
on the problem did not exist. And yet so rapid is the progress 
that has since been made that there is published in the current 
number of "Education" a study of that problem based on the 
school histories of more than 25,000 children. 

From a Michigan city there comes a striking illustration of the 
degree to which we as educators have enjoyed that freedom which 
comes through being entirely unhampered by facts. About five 
years ago a movement gained headway in that city for the es- 
tablishment of kindergartens. The advocates of the proposed 



innovation gave as their most weighty argument the claim that 
children who pass through the kindergarten subsequently com- 
plete the elementary grades in less time than do those who have 
not enjoyed the advantages of such training. The faction oppos- 
ing the establishment of the kindergartens denied the validity 
of this argument. To settle the question the school authori- 
ties wrote to school superintendents all over the country asking 
whether children who had gone through the kindergartens sub- 
sequently completed the work of the grades more quickly than 
did those who had not received such training. Replies were 
received from the superintendents of 72 cities. Of these, 49 
answered that they thought that children having kindergarten 
training subsequently made more rapid progress than the others, 
but that they did not know. The other 23 replied that they 
held the opposite opinion but that they did not know. 

That result was typical of the supremacy of speculation over 
evidence in education. In this problem, as in other problems, 
opinions have varied. There has been a consensus of belief but 
there has been an almost absolute absence of definite knowledge. 
Kindergartens have been increasingly numerous in America 
since Elizabeth Peabody established the first one in Boston in 
1868. They now exist by the thousands and on them we have 
spent each year hundreds of thousands of dollars. During the 
entire period a favorite argument in their support has been the 
one relied on to secure their establishment in the Michigan city, 
and yet until recently no one has been able to state in definite 
terms anything about the real effect of kindergarten training. 

This situation no longer exists for within the past three years 
extensive investigations have been conducted comparing the 
school records of many thousands of children who have had 
kindergarten training with the school records of the children in 
the same systems who have not had kindergarten training. 

Educational Surveys 
The new scientific method has not been confined to the in- 
vestigation of isolated problems. In city after city across the 
country its aid is being invoked to evaluate educational results 
through the medium of the school survey. Unheard of only a 
few years ago, these city-wide educational inquiries have been 
made, or are in progress, in such cities as Boston, Baltimore, Boise, 
Montclair, Orange and New York. Already they are being 



7 
planned for in other localities, and one embracing the system of 
the entire state of Wisconsin is now under way. 

Conservatism versus Progressivism 
The progress of this educational revolution has been stoutly 
contested and each forward step has been greeted by an anvil 
chorus of opposition in which the notes ranged from the grudging 
admissions of the skeptic to the fiery denunciations of the edu- 
cational reactionary. Always retiring and always fighting, these 
forces of opposition have abandoned as untenable their con- 
tention of fifteen years ago that any and all attempts at measure- 
ment in education are silly and dangerous. Having given up this 
position they next took refuge in the firm declaration that while 
material matters in education may be quantitatively investi- 
gated, the immaterial problems of the teaching process can never 
be submitted to such treatment. They admitted that it would 
probably do no harm to discover the more important facts with 
respect to financial expenditures and the progress of pupils, but 
firmly declared that no phase of intellectual phenomena would 
yield to statistical analysis. 

The Measurement of Educational Products 
No sooner was this doctrine fully formulated than there 
appeared a set of scientific students of education presenting 
measuring scales with which to gauge the performance of the 
children in their classroom work. Thorndike with his measuring 
scales for handwriting, Stone and Courtis with their standardized 
tests in arithmetic, and Hillegas with his method for measuring 
the quality of English composition again forced the champions 
of tradition to retire and find a new point of defense. 

Character and Efficiency 

The final citadel in which the old guard is now making its 
last stand consists of the objection that the most important 
elements of true teaching can never be measured. 

They claim, and they are right in claiming, that we can never 
determine by mathematical measurement the degree to which the 
strong man and the noble woman influence for good the characters 
of their pupils. But what they overlook is the fundamental 
truth that in education, as in other pursuits of life, character and 
efficiency go hand in hand. As school executives make practical 



8 

application of the newer scientific tests, no fact stands out with 
more impressive distinctness than that the teachers whose classes 
make the best records are the teachers who are the most truly 
successful in the shaping of character. 

Individual Development not Universal Uniformity 
There remains one other objection, less frequently advanced 
but sometimes voiced, and that is that the advocates of the 
scientific method aim to reduce all work in education to the dead 
level of uniform precision. This charge is born of a complete 
misunderstanding of the ends, aims, and processes of the new 
method. Its aim is not uniformity but individual development. 
The measured beat of the concert recitation is not music in the 
ears of the scientific students of education. The sight of a rigid 
row of reciting children with toe tips nicely adjusted to a line 
painted on the classroom floor does not cause their souls to leap 
in admiration. Their ideal of school discipline does not consist 
of having a roomful of growing children accomplish the amazing 
feat of sitting through an entire period without moving a muscle 
or winking an eye. Their ideal of educational administration 
does not contemplate a uniform country-wide daily program by 
which each recitation period in every city and hamlet shall be 
fixed by a master clock located at the seat of the National Gov- 
ernment in Washington. 

The Scientific Method Means the Measurement 
of Results 

The object of the new method is the substitution of evidence 
for opinion and knowledge for speculation. Its champions are 
working to develop measurements in education because they 
realize that only by this method can education become an art 
and a science and its practice be changed from a vocation to a 
profession. They scan the history of science and remember that 
through the development of measurements astronomy grew out 
of astrology, chemistry emerged from alchemy, and physics de- 
veloped from mystery. 

They read the history of education and realize that the as- 
tonishing progress of the past decade has come from shifting the 
form of inquiry from asking "What results can or might we get?" 
to "What results are we getting?" This makes the pupil and 
not the teacher the center of interest. It calls a halt on the futile 



9 
quest for standards of attainment on which we have never come 
to an agreement, and aims instead to discover units of measure- 
ment. Simple as it sounds, this change from asking "What 
results should we get?" to "What results are we getting?" is the 
keynote of the whole scientific method in education. To answer 
the question in its new form means the development of units of 
measurement, and when these are secured the standards of at- 
tainment will work themselves out automatically. 

The Future 
The certainty about the scientific method in education is that 
it is with us. That it will develop enormously in the immediate 
future is entirely sure. What its effects will be we can as yet 
only surmise. The dangers involved are as real and imminent 
as the advantages are self-evident. These dangers will arise 
from the mass of superficial and erroneous results that will 
certainly be presented to the educational world in the guise of 
scientific contributions to applied pedagogy. What is to be our 
attitude toward each new contribution? 

My own answer is that we must welcome them all, but chal- 
lenge them all, and attempt to verify them all. Every figure, 
every process, and every conclusion, whether presented by the 
educational expert or the novice, must be submitted to the most 
rigid scrutiny and searching analysis before being accepted as 
worthy of inclusion in the new pedagogy. 

In proportion as we are thus enabled to retain the genuine and 

to reject the spurious, education will move forward among the 

other sciences. Its new methods will substitute knowledge for 

peculation and evidence for opinion. Its marshalled facts, 

•pressed in definite terms, will demolish the hosts of legend, 

oerstition, tradition, and theory. 

Inder the new regime the studies to be included in the curric- 
l and the methods by which they are taught must have a 
valid reason for being than the fact that our forefathers had 
in their schools. 

w much?" and "How many?" and "With what results?" 

lg to displace guess-work, imagination, and oratory as 

f or shaping educational policies. The old method has 

'ation within the sheltering walls of the cloister in which 

nal peep-hole has been cut to satisfy the parent and 

tax-payer. The new method proposes education in the 

der the clear and penetrating rays of the search-light. 



MAY 11 \M 



Some Publications on Measurements in Education 
Issued by the Division of Education, 
Russell Sage Foundation* 



No. 61. The Relation of Physical Defects to School Progress 
Leonard P. Ayres, Ph.D. 
A statistical study based on 7608 cases. 9 pp. 
No. 107. The Binet-Simon Measuring Scale for Intelligence: Some 
Criticisms and Suggestions. Leonard P. Ayres, Ph.D. 
A critical study of these tests and suggestions as to their 
adaptation to our conditions. 
No. 108. The Identification of the Misfit Child 
Leonard P. Ayres, Ph.D. 
Data from a study of the age and progress records of school 
children in twenty-nine cities. 
No. no. The Relative Responsibility of School and Society for the 
Over-age Child. Leonard P. Ayres, Ph.D. 
Data from a study of the age and progress records of school 
children in twenty-nine cities. 
No. 112. The Relation Between Entering Age and Subsequent Prog- 
ress Among School Children. Leonard P. Ayres, Ph.D. 
Evidence from three investigations of the problem "What 
is the best age to send a child to school?" 
No. 113. A Scale for Measuring the Quality of Handwriting or 
School Children. Leonard P. Ayres, Ph.D. 
A quantitative study of legibility. (Report five cents. Scr 
five cents.) 

• A charge of five cents a copy is made for these publications unless otherwise specified. 



No. 107 



Health, Education, Recreation 



THE BINET-SIMON MEASURING SCALE 
FOR INTELLIGENCE: SOME CRITI- 
CISMS AND SUGGESTIONS 



LEONARD P. AYRES, Ph.D. 




Reprinted from 
The Psychological Clinic, November 15, 191 1 

BY THE 

Department of Child Hygiene 
Russell Sage Foundation 

400 Metropolitan Tower, New York City 



u-ii-16 



I 

DEI 



LIBRARY OF CONGRESS 



'■ii'ii'i'iiiimiiiiiiiniiiiii in | || i 

021 338 617 7 ft 



